Connectionist Models in Multimodal Human-computer Interaction
نویسندگان
چکیده
We present an overvi ew of our l aboratori es' research on Mul timodal Human-Computer Interf aces. By expl oi ti ng al l avai l abl e channel s of human communi cati on we aim to i ncrease exi bi l i ty, robustness, and natural ness of human-computer i nteracti on. The i nformati on sources we process i ncl ude Speech-, Character, and Gesture Recogni ti on, Faceand Eye Tracki ng, Li preadi ng, and Sound Source Local i zati on. Connecti oni st and hybri d techni ques are used throughout.
منابع مشابه
Face Locating and Tracking for Human{Computer Interaction
E ective Human-to-Human communication involves both auditory and visual modalities, providing robustness and naturalness in realistic communication situations. Recent e orts at our lab are aimed at providing such multimodal capabilities for humanmachine communication. Most of the visual modalities require a stable image of a speaker's face. In this paper we propose a connectionist face tracker ...
متن کاملThe MADE Framework: Multimodal Software for Affective Education
This paper proposes a framework for multimodal educational systems, considering the affective strategies. The ability to communicate emotionally and cognitively plays an important role in human-computer interaction (HCI) and education. The challenge is how theoretical models of HCI can inform affective multimodal education.
متن کاملUsing critical path analysis to model multimodal human-computer interaction
In this paper, the concept of multimodal human-computer interaction is explored. It is proposed that multimodality can be de"ned from human or technology perspectives, which place emphasis on di!erent attributes of the system. Furthermore, in this paper it is argued that the most e!ective de"nition of multimodality concentrates on task and goal dependencies. Not only does this permit considerat...
متن کاملHuman-centric framework for perceptually adaptive interfaces
Multimodal interfaces have long held the promise of enhanced and effective human machine interaction. The ultimate goal of multimodal interfaces is to facilitate human activity allowing seamless exchange of information. This goal requires a coordinated development effort that incorporates a thorough understanding of human perceptual system in the design of interfaces. In this manner, multimodal...
متن کاملTowards Ambient Intelligence: Multimodal Computers that Understand Our Intentions
This paper discusses the state-of-the-art and future developments of conversational multimodal interaction with computers, one of the corner stones of all future eCommerce and eWork applications. On the basis of past and present experience it is argued that natural, conversational interaction can only be achieved on the basis of powerful cognitive models of multimodal human-machine interaction....
متن کامل